The team at Charité – Universitätsmedizin Berlin, TU Berlin, and the University of Oslo, have developed the system that, for the first time, integrates morphological, molecular, and histological data in a single analysis. The system also provides a clarification of the AI decision process in the form of heatmaps. The heatmaps show which visual information influenced the AI decision process and to what extent, which enables doctors to understand and assess the plausibility of the results – representing an essential step forward for the future use of AI systems in hospitals. The research has been published in Nature Machine Intelligence. The molecular characterisation of tumour tissue samples is becoming increasingly important for cancer treatment, with studies being conducted to determine changes to DNA as well as the gene and protein expression in the samples.

Universitätsmedizin Berlin and TU Berlin as well as the University of Oslo have developed a new tissue-section analysis system for diagnosing breast cancer based on artificial intelligence (AI). Two further developments make this system unique: For the first time, morphological, molecular and histological data are integrated in a single analysis. Secondly, the system provides a clarification of the AI decision process in the form of heatmaps. Pixel by pixel, these heatmaps show which visual information influenced the AI decision process and to what extent, thus enabling doctors to understand and assess the plausibility of the results of the AI analysis. This represents a decisive and essential step forward for the future regular use of AI systems in hospitals. The results of this research have now been published in Nature Machine Intelligence.